DOCUMENT RESUME 

ED 385 950 EA 026 985 



AUTHOR 
TITLE 

INSTITUTION 
PUB DATE 
NOTE 



AVAILABLE FROM 



PUB TYPE 



Newman , Lynn 

School-Agency-Community Partnerships: What Is the 
Early Impact on Student School Performance? 
SRI International, Menlo Park, Calif. 
Apr 95 

36p.; Paper presented at the Annual Meeting of the 
American Educational Research Association (San 
Francisco, CA, April 18-22, 1995). The evaluation 
presented is being conducted for the California 
Partnership for School-Linked Services and funded by 
the Foundation Consortium for School-Linked 
Services . 

SRI International, 333 Ravenswood Avenue, Room BS178, 
Menlo Park, CA 94025 (Order No. 206, free; quantity 
price $2 per copy) . 

Speeches/Conference Papers (150) — Reports - 
Research/Technical (143) 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MF01/PC02 Plus Postage. 

'^Academic Achievement; Agency Cooperation; Elementary 
Secondary Education; ,v Family Programs; Family School 
Relationship; Human Services', ^Integrated Services; 
,v Outcomes of Education; Partnerships in Education; 
School Health Services; '-Social Services 
"''California; "''Healthy Start Program CA 



ABSTRACT 

California's Healthy Start initiative, begun in 1991, 
awards grants to local groups of schools and public and private 
organizations to collaboratively plan and implement strategies to 
integrate services for children and families, which are provided at 
or through the school. This paper presents findings of a study that 
examined whether participation in the Healthy Start program 
contributed to improved educational performance, specifically, as 
indicated by students' grade-point averages (GPAs) . The school 
records of 268 students were reviewed for the years 1991-92 and 
1992-93. Student performance befcre and after program participation 
was compared. To identify the contribution of services to changes in 
students' GPAs, four separate analyses were conducted for each of the 
four types of educational services — tutoring, counseling, advocacy, 
and parent education. Findings indicate that educationally oriented 
services contributed to small gains in school performance even after 
relatively short-term participation in those services. Students from 
families who needed basic assistance (food, clothing) were less 
likely to achieve gains than were students from families with fewer 
needs. The provision of basic-needs services through Healthy Start 
helped to reduce barriers to learning presented by deficits in such 
basics as food and clothing. The Healthy Start program had positive 
education effects for children and families, although effects at this 
early stage were small and often not statistically significant when 
differences between students were controlled. Eight tables and two 
figures are included. Appendices contain an overview of 
data-collection methods and a list of related reports. (Contains nine 
references.) (LMl) 
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School-Agency-Community Partnerships: What Is the 
Early Impact on Student School Performance?' 



A growing number of communities and states are attempting to restructure their 
systems of education, health, mental health and social services to better meet the needs of 
children and families (e.g., see U.S. GAO, 1993, Adler and Gardner, 1994, Gomby and 
Larson, 1992). Partnerships and inter-organizational collaboration are central to these 
restructuring strategies. The intent of these changes is to create a new delivery system form 
the fragments of the existing multiple systems that serve children and families — a "pro family 
system" that encourages institutions and community organizations to work together to develop 
integrated, comprehensive, and high-quality services that support and strengthen families. 
Ultimately, the goal of this reorganization of educational, social and health resources is to 
ensure that children, youth, and families receive the services that they need to improve the 
outcomes of childhood and adolescence. 

California's Healthy Start initiative, launched in 1991 under the auspices of Senate Bill 
620, is one example of these strategies for achieving systems change in inter-organizational 
collaboration. The Healthy Start Support Services for Children Act authorized the awarding of 
grants to local groups of schools and public and private organizations, in collaboration, to plan 
and implement strategies to integrate services for children and families and to provide them at 
or through the schools. In the initiative's first year, the California Department of Education, as 
the lead agency for Healthy Start, used an appropriation of $20 million form the legislature to 
award 40 grants of up to $400,000 each to local collaboratives that demonstrated in 
competitive grant applications that they were ready to implement comprehensive, integrated, 
school linked services. These 40 operational sites throughout the state were granted funds for 
a three year period to integrate the education, health, mental health, and social service 
delivery systems in their communities so that children and families would have readier access 
to a broader array of services through their schools. At the same time, SRI International was 
commissioned by the California Department of Education and the Foundation Consortium for 
School-Linked Services to conduct a 3-year evaluation of the processes and outcomes of the 
efforts of these operational grantees. 



' Based on analyses and reporting in A Healthy Start fcr California's Children and Families: Early 
Findings from a Statewide Evaluation of School-Linked Services. Prepared by Mary Wagner, 
Ghari Golan, Debra Shaver, Lynn Newman, Marjorie Wechslerand Fiona Kelley. 
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While a wide variety of benefits are expected to accrue to participants in Healthy Start, 
improving students' school performance is an important objective of Healthy Start; as stated in 
the SB 620/Healthy Start RFA, "the goal of Healthy Start is to produce measurable 
improvement in outcomes for students.. .in the areas of school attendance and performance..." 
(California Department of Education, 1992a, p.7). Improving educational outcomes is an 
objective that has guided the activities of most Healthy Start programs, 83% explicitly stated a 
goal of improving educational performance, and one-fourth of all Healthy Start services were 
academically related. This paper examines the impact of integrated services on student 
school performance. Specifically, we address the following questions: 

• What changes in educational outcomes are experienced by Healthy Start core clients? 

• What changes in educational outcomes are achieved by students in programs with a 
stated goal of improving educational performance? 

• Were some program models particularly effective in improving educational performance? 

• Did improvements in educational outcomes accrue to particular kinds of students? 

• What was the relationship between receiving educational services, such as tutoring or 
school counseling and educational outcomes? 

To answer these questions, this paper focuses on the school performance of a subset 
of students in Healthy Start programs, those students who were most intensively served by 
their Healthy Start program. The paper begins with a description of schools and students 
participating in Healthy Start. It then continues with an examination of the changes in 
educational outcomes of a subset of students in schools associated with Healthy Start 
programs, comparing school performance, (attendance, teacher ratings of student behaviors 
and grades), before and after student involvement in the Healthy Start program. Educational 
outcomes will be considered first for core Healthy Start students overall, and then for core 
students in different types of Healthy Start programs, and with different individual 
characteristics. The paper will then focus on an exploration of the relationship between 
receiving particular Healthy Start services and school performance. 

Data Sources 

Findings focus on the first 40 sites that were awarded Healthy Start grants in 1992. 
Analysis presented here pertain to a subset of Healthy Start students, students who are 
considered by their Healthy Start site to be core students— those most intensively served by 
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Healthy Start programs. School records for the 1991-92 and 1992-93 school years were 
collected for the "index child" in core families— the student through whom the family initially 
became involved with Healthy Start— if that child had completed the Healthy Start intake process 
by April 15, 1993. This date was selected to ensure that students would have been in school for 
at least one quarter of the school year after beginning their involvement in Heaithy Start. 

School records were requested for 675 students. Although 545 school records were 
returned (a response rate of 81%), only 268 of the returned school records had sufficient 
before and after Healthy Start involvement to be included in this analysis; school records often 
included only partial information, usually due to high student mobility. School performance 
analysis includes only those students who had a minimum of a full grading period of school 
performance information both before and after their initial involvement with the Healthy Start 
program. The period of time for which records were obtained before intake into Healthy Start 
varied; on average data were obtained for about 1.25 school years before Healthy Start 
enrollment and for just under 1 semester after Healthy Start enrollment. 

It is important to note that because 1993-94 school records have not yet been 
collected, we have school records for only a short period of time after students became 
Healthy Start core clients. It is early to see the impact of Healthy Start involvement, and this 
analysis should be considered preliminary until the 1993-94 school information is collected and 
included in the analysis. 

Descriptions of schools participating in Healthy Start are based on the Consolidated 
Programs Description Database, a database, managed by the California Department of 
Education, which contains information form a number of smaller databases, and on mail 
surveys of principals in Healthy Start schools. 

Descriptions of Healthy Start programs are derived form Healthy Start grant proposals, 
visits to Healthy Start programs, telephone interviews with grant coordinators, telephone interviews 
with up to five collaborative partners, and mail surveys with remaining collaborative partners. 

Services provided by Healthy Start programs are documented through several forms, 
varying with the type of service being delivered. These forms include, event description forms, 
event attendance sheets, and service encounter logs. These forms varied with the type of 
service being delivered. Family background information was documented through the family 
intake form. 



Healthy Start Programs and Schools 

There is no single model of services being implemented among the Healthy Start 
grantees; rather, Healthy Start reflects the view that "a community must develop an approach 
and tailor program design to capitalize on its particular strengths and opportunities and to 
respond to its citizens unique combination of needs and expectations" (Levy and Shepardson, 
1992). Healthy Start grantees are self-generated collaboratives that are planning and 
implementing services that they believe will meet the needs of the students and families they 
have targeted. The program designs range broadly, form intensive, family-centered programs 
at single schools that involve ongoing relationships between family service advocates and 
client families, to district-wide screening and referral programs that served multiple schools. 
Programs also varied in their use of school versus outside service personnel. For example, 
some programs used professionals form community-based organizations and interns to staff 
on-campus clinics and centers, whereas other programs relied heavily on school and district 
personnel and on existing coordination mechanisms, such as student study teams. Finally, 
some programs were focused on specific target populations, such as adolescents or new 
district enrollees, whereas other programs offered services to all students and family members 
within their schools. 

Healthy Start programs can be grouped into four basic types. The first three types, 
which focus primarily on families and young children (i.e., those in elementary school), are 
distinguished mainly in how consumers enter the service delivery system. The first typs of 
program, family resource center, is distinguished by having an. identifiable space on a school 
campus to which consumers can come on their own initiative. The distinguishing feature of 
satellite family service centers, the second type of program is the existence of a center located 
somewhere other than at an operating school, where families can go to obtain a variety of 
services. The third type of program, family service coordination teams, is known less as a 
place than as a function. In these programs, the primary way in which consumers gain access 
to services is through referrals to a multidisciplinary team of service professionals. Although 
most of the programs in the other categories have such teams, the central role of these 
multidisciplinary teams in how clients enter the service delivery system distinguishes them form 
other programs. The fourth group, youth service programs, includes programs that focus 
primarily on the needs of adolescents. In summary the typology of school-linked services 
programs includes the following groups: 
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School-site family resource centers. The most common model of Healthy Start program 
is the family resource center at a school. This is an identified space at an operating 
school to which families can come for a variety of services on their own initiative or by 
referral form teachers, other service providers, or a service coordination team. Sixteen 
of the projects fit this model, and all of them are located at elementary schools. Most of 
the family resource centers (12) serve families form a single school; however, 4 of them 
serve multiple schools, including 2 programs that serve families form a neighboring 
junior high and high school. 

Satellite family service centers. Four sites involve a service center that is not located at 
an operating school site, a fact that potentially has implications for the breadth of 
clients the center can serve and its accessibility to clients. These satellite service 
centers vary in the extent to which they provide services directly to clients at the center. 
Two of the centers emphasize identification of family needs and referral to community 
resources, rather than incorporating a strong component of direct service provision, 
whereas two include more direct services. These projects serve more than one school; 
two are districtwide programs. 

Family service coordination teams. This model does not base its services around any 
physical location; rather, it features a team of service professionals who review the 
needs and status of students who have been identified as at risk or whose parents 
have asked for help. The team often is an outgrowth or extension of the student study 
team that functions in most schools. Through Healthy Start, these teams often have 
added new members to extend the range of services and expertise that can be 
obtained in support of particular children and families. Members of the team can 
function as direct service providers to a child or family. Twelve projects generally fit this 
model, with 10 involving elementary schools only and 2 including one or more junior 
high or high schools. Seven of these sites involve a single school, while five are 
multischool projects. 

Youth service programs. Eight projects focus their services on addressing the health, 
education, and social needs of adolescents. Five projects organize themselves around 
a school-based or school-linked student health clinic. Two other programs are 
identified by their target clientele — pregnant teens and school-age parents and their 
children — while one program serves a broader population of secondary school youth. 
Six of the teen support sites operate at and serve single schools, one site provides 
services at three secondary schools, and one adolescent health clinic is located off 
campus and serves two nearby secondary schools. 

Despite their differences, all Healthy Start programs shared the common experiences 
of trying to maintain interagency collaboration and of providing school-linked services that 
focused on serving the combination of educational, health, mental health, and social service 
needs of their participants. In this way they are similar to other coliaboratives for school linked 
services (Mattessich and Monsey, 1992; Melaville and Blank, 1993). Moreover, the goals of 
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most Health Start programs reflected the principles of the initiative — to create a service system 
that is more outcome focused, comprehensive, integrated, accessible, family focused, 
preventive and culturally appropriate. 

Healthy Start Schools 

All Healthy Start programs are linked to schools. Although the 40 operational programs 
involve 124 schools, the majority of programs (23) operate at single schools. Most of the 
others operate in groups of 2 to 5 schools, and three programs are district- or county-wide, 
involving more than 10 schools each. 



Table 1 

CHARACTERISTICS OF HEALTHY START SCHOOLS 

Percentage of 
Healthy Start Schools 



School level 

Elementary 75 

Intermediate 8 

High 14 

Other (special education school, juvenile hall, countywide/ 3 
districtwide program, public alternative/opportunity program) 

Enrollment 

Small (fewer than 250 students) 21 

Medium (250 to 499 students) 21 

Large (500 to 999 students) 36 

Very large (1 ,000 or more students) 22 

Metropolitan status 

Urban 29 

Suburban 42 

Rural 29 

N 124 
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Healthy Start grant funds have gone to schools that represent all levels and enrollment 
sizes and all different types of geographic areas. Three quarters of the schools are 
elementary schools, reflecting the distribution of schools in the state as a whole (Market Data 
Retrieval, 1990) (Table 1). The enrollment in these schools ranges form very small to very 
large. The smallest school is in a rural one-school district with 19 students that is part of a 
countywide program operating in 25 schools. The largest Healthy Start school is an urban 
high school with 3,263 students. Healthy Start schools are located in urban, suburban, and 
rural areas, with suburban schools having higher representation than others (42% vs. 29% 
each for urban and rural schools). 



Table 2 



CHARACTERISTICS OF HEALTHY START SCHOOLS RELATIVE 
TO GRANT ELIGIBILITY CRITERIA 



Percentage of 
Healthy Start Schools 



Proportion of student body: 



Receiving Aid to Families with Dependent Children 



0% to 24% 
25% to 49% 
50% to 74% 
75% to 100% 



64 
27 
7 
2 



With limited English proficiency 



0% to 24% 
25% to 49% 
50% to 74% 
75% to 100% 



48 
24 
19 
9 



Eligible for free or reduced-price meals 



0% to 24% 
25% to 49% 
50% to 74% 
75% to 100% 



16 
15 
43 
27 



N 



124 



0 
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Despite the diversity of Healthy Start schools, many students and families at Healthy 
Start schools had serious and multiple needs, as implied by their high poverty levels, low 
English proficiency, and high academic needs (Table 2). To qualify for funding, schools must 
have 50% or more of their students either receiving Aid to Families with Dependent Children, 
or having limited English proficiency, or being eligible to receive free or reduced lunch. 

The students attending Healthy Start schools represent diverse ethnic backgrounds. 
More than half of the schools (51%) had student bodies in which the majority were children of 
color. Of those schools with the highest percentages of children of color, many were 
comprised largely of Latino students. Thirteen percent of all Healthy Start schools had a 
Latino population of 75% or greater, 6% of schools had a Latino population of more than 90%. 

The mobility rates of students attending Healthy Start schools can be quite high, 
hampering the ability of students to have a consistent, integrated educational experience or to 
establish social bonds at their school. In nearly one-third of Healthy Start schools (31%) one- 
quarter to one-half of students moved away during the 1931-92 school year. In 10% of the 
schools, more than half of the student population moved away during the school year. 

The academic needs of students attending Healthy Start schools can also be quite 
high. The percentage of students enrolled in Chapter 1 compensatory education programs 
provides some insight into the general achievement levels of the students participating in 
Health Start. Students are identified for services in Chapter 1 schools based on their 
educational achievement. Forty percent of Healthy Start schools had at least half of their 
students enrolled in Chapter 1 . Fourteen percent had three-fourths or more of their student 
population enrolled in Chapter 1. 

Educational Outcomes of Core Healthy Start Students 

Healthy Start students clearly faced several obstacles to their successful performance 
in school. Many are poor, with limited English skills, high mobility rates and high academic 
needs. As presented in Table 3, almost one third of the families of core students with school 
records received AFDC (29%) and almost one third had limited English proficiency (31%). 
Healthy Start programs were designed to begin addressing some of the needs students bring 
with them to school. This section focuses on changes in student school performance, before 
and after their involvement in Healthy Start 
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Table 3 



CHARACTERISTICS OF HEALTHY START CORE STUDENTS 
WITH SCHOOL RECORDS 

Percentage of core students with 
school record information 



Ethnicity 

American Indian <1 

Asian 21 

African-American 1 5 

Latino 50 

Pacific Islander <1 

White 14 

Grades 

K-3 36 

4-8 24 

9-12 40 

Gender 

Male 55 

Female 45 

Family receives AFDC 29 

Limited English proficient 31 

N 268 



Three measures of school performance: teacher ratings of student behaviors, 
attendance, and grades, will be described here Research suggests that these aspects are 
intimately related. As a measure of student engagement in the educational process, high 
absenteeism is noted as onei of the most powerful predictors of poor grades and early school 
leaving (Wagner, Blackorby, and Hebbeler, 1993). Without meeting the minimal expectation of 



attending school, students cannot benefit form their educational experiences. Yet high mobility 
and poor health among Healthy Start families often leads to high absenteeism. Students' 
behaviors in class also can influence performance, particularly as reflected in teacher 
judgments regarding classroom grades, tt is often in this behavioral aspect of performance 
that individual and family stresses first are evidenced in the classroom. Finally, student grades 
are highlighted as a summary measure of academic achievement. 

Educational outcomes are presented as they pertain to core students — those most 
intensively served by Healthy Start programs. These are the students for whom the greatest 
impact might be expected; educational impacts reported here should not be assumed to result 
for all Healthy Start students. Indeed, as indicated earlier, the students for whom we have 
information are only a sample of all core students. Not all school records returned by schools 
had a full grading period both before and after the student's participation in Healthy Start, a 
requirement to be included in this analysis. It is also important to note that we have school 
records for only a short period of time after students became Healthy Start core clients. It is 
early to see the impact of Healthy Start involvement, and this analysis should be considered 
preliminary until the 1993-94 school information is collected and included in the analysis. 
Given this understanding of who core clients were and the information available about them, 
the following sections highlight changes in educational outcomes for Healthy Start core 
students. 

Student Behavioral Performance 

Many students, particularly those in elementary school, receive grades or other forms of 
teacher ratings of their behavior. Behavior ratings in Healthy Start elementary schools 
generally focused on conduct and study habits. Conduct factors included such behaviors as 
self-control, working well with others, being respectful and considerate, having a positive 
attitude, and following class and school rules. Study skills factors included working 
independently, completing homework, following directions, participating in discussions, staying 
on task, and using time in a productive way. The variety of forms of grades students received 
for these two types of behaviors were converted to a 3-point scale, with a value of 3 for 
excellent or good behavior, 2 for satisfactory behavior, and 1 for behavior deemed to need 
improvement or that was unsatisfactory. An average student behavior performance score was 
then created, combining grades received for conduct and study skills. 
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Table 4 



STUDENT BEHAVIORAL PERFORMANCE, BY QUARTILE 



Ratings before 
Healthy Start 



Ratings after 
Healthy Start 



Change in 
Ratings after 
Healthy Start 



n 



All students with ratings 
Initial behavioral quartile 



1.97 



2.04 



.07+ 



106 



Lowest 
Mid/low 
Mid/high 
Highest 



1.34 
1.79 
2.08 
2.57 



1.55 
2.00 
2.14 
2.42 



.06 
-.15 



.21* 
.21*** 



24 
20 
37 
25 



+p<.10; * p<.05; ***p<.001. 



Overall students' behavior ratings increased marginally after enrollment in Healthy Start 
programs, form a mean of 1.97 to a mean of 2.04 (p<.10). Yet these average ratings obscure 
the fact that students varied widely in itieir conduct and learning styles, some performing poorly 
while other surpassed teacher expectations. Table 4 shows elementary school children that 
had behavior ratings divided into quartiles based on their ratings and shows the difference in 
behavior scores given before and after their enrollment in Healthy Start. Students who 
exhibited the poorest behaviors made the largest improvements, with children in both the lowest 
and mid-to-low behavioral quartiles increasing their behavior ratings by .21 (p<.05 and .001). 

Attendance 

Because many schools keep attendance information only for the current school year, 
the attendance analysis is limited to school record information collected in the 1992-93 school 
year (the school year in which most 1 992 Healthy Start operational grantees began to enroll 
core clients throughout the spring), while other school analyses include 1991-92 as well as 
1992-93 information. Thus, attendance before Healthy Start is for a period of 1 semester to 3 
quarters, whereas information after Healthy Start if for 1 quarter to 1 semester. Further, 
schools vary widely in the time period they use to report attendance data. To create a uniform 
reporting period for all schools, actual absentee information has been prorated to reflect the 
number of days a student would have been absent per school year. 




With absentee rates of 12 and 13 days per school year, there was no significant 
difference in school attendance before or after Healthy Start involvement. Because our 
attendance information for most students is for less than 1 semester after they became 
involved with Healthy Start, it is early to expect these data to show impacts on attendance 
rates. Further, it also is important that if a student's intake date fell within the first third of a 
grading period, that grading period was considered to be "after Healthy Start." Thus, for those 
receiving semester grades, up to 2 months of attendance information included in the after- 
Healthy-Start category might actually have occurred before Healthy Start enrollment. It is 
possible that poor attendance during these few months just before intake wes one of the 
precipitating reasons for the student's being referred to Healthy Start. Because at this time we 
are unable to completely distinguish attendance before and after Healthy Start, it is premature 
to analyze absentee rates fully or to conclude anything about impacts on them. When the 
evaluation obtains student records for a longer period of time after Healthy Start enrollment, 
more definitive analyses will be possible. 

Grades 

Schools vary widely in the metrics they use to grade students' academic performance. 
Some grades involved checks, pluses and minuses, whereas others are on a 5-point scale of A 
through F. To create a consistent grading scale across schools, grades for elementary and 
middle school students were converted to a 3-point scale, where 3 was assigned to outstanding 
or good grades (the equivalent of A or B), 2 to satisfactory grades (the equivalent of C), and 1 
to grades that needed improvement or were unsatisfactory (the equivalent of D or F). Grades 
for high school students were more uniform and often corresponded to the traditional 5-point 
scale, where 4 is assigned to an A, 3 to a B, 2 to a C, 1 to a D, and no credit to an F. Where 
the metric of high school grades differed form this, grades were converted to this scale. Mean 
performance scores, or grade point averages (GPAs), were then created for each student for 
the interval of time in the 1991-1993 school years before the student became a Healthy Start 
core client, and for the period following the student's involvement in Heal ly Start. 
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Table 5 



STUDENT GRADE PERFORMANCE (GPA), BY QUARTILE 



GPA before GPA after Change in GPA 

Healthy Start Healthy Start after Healthy Start n 

Overall 2.08 2.15 .07* 268 

Initial GPA quartile 

Lowest 1.26 1.47 .21* 67 

Mid/low 1.83 1.87 .04 66 

Mid/high 2.27 2.43 .07 68 

Highest 2.90 2.93 .03 67 



* p<.05; **p<.01. 



As presented in Table 5, grades showed a marginal, though statistically significant 
improvement after students became Healthy Start core clients, with the average GPA for all 
students increasing form 2.08 to 2.15 (p<.05). As indicated earlier, we have school records for 
only a short period of time after students became Healthy Start core clients (often only 1 
quarter); yet, surprisingly, even in that short time, positive gains in students' grades are noted. 

Further, Table 5 indicates that the strongest gains were clearly made by those 
struggling the most with grade performance. Students in the lowest grade quartile experienced 
a .21 gain in GPA (p<.05), a gain 7 times greater than that experienced by those in the highest 
quartile. 

Although it is possible that this improvement results form "regression to the mean" — the 
tendency for more extreme values measured at one point in time to moderate or tend back to 
the average value when measured a second time — the GPA measured for core students 
before their involvement in Healthy Start generally was averaged over more than one grading 
period, indicating that poor grades were not a one-time measurement phenomenon but a more 
pervasive problem of poor performers. Thus, a change in that pattern of poor performance is 
unlikely to occur by chance, but is more likely to result form direct intervention. Regression to 
the mean also would affect all poor performers equally, resulting in uniform changes across 
grade levels, for example; findings are not consistent with this scenario. 
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Here we've examined overall changes school performance for Healthy Start core 
students. The following section details variations in student grade point average (GPAs) 
associated with different kinds of Healthy Start programs. Student grades are highlighted as a 
summary measure of academic achievement. 

Individual Variations 

To address the question of whether some students were experiencing greater 
educational benefits form involvement with Healthy Start, we have examined changes in 
grades by school grade level, student gender and ethnic background. 



Table 6 

STUDENT GRADE PERFORMANCE (GPA), BY STUDENT CHARACTERISTICS 



GPA before 
Healthy Start 



Student grade level 

K through 8 1.97 

9 through 12 2.22 

Gender 

Male 1.96 

Female 2.24 

Ethnic Background 

African-American 1.75 

Asian 2.65 

Latino 1.96 

White 2.05 



* p< 05; **p<01 



GPA after Change in GPA 
Healthy Start after Healthy Start n 



2.07 .10** 159 

2.67 .05 109 

2.06 .10* 138 

2.25 .01 119 

1.82 .08 41 

2.72 .07 54 

2.01 .05 128 

2.20 .15 34 
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As presented in Table 6, students in the primary grades experienced greater growth in 
their grade performance than did high school students; with students in grades K-8 exhibiting a 
10 point gain (p< 01), compared with a 5 point gain in GPAs for those in high school. Most of 
the educational growth experienced by those in grades K-8 were experienced by those in the 
earliest grades. For example, those in grades K-3 showed a 15 point gain, while those in 
grades 4-8 only experienced a 1 point gain. 

While girls had an overall higher GPA than boys, both before and after becoming 
Healthy Start core clients, boys experienced larger gains in their GPAs. Boys exhibited a 10 
point gain (p<.05), while girls only experienced a 1 point gain. 

Although student GPAs, both before and after becoming Healthy Start core clients 
differed by student ethnic background, student gains in GPA did not differ significantly for 
students with different ethnic backgrounds. 

Program Variations 

As indicated earlier, although no two Healthy Start programs were exactly alike, they 
can be grouped into four basic categories: 

School-site family resource centers, (n=16) had an identified space at an operating 
school to which families could come for a variety of services on their own initiative or by 
referral form teachers, other service providers, or a service coordination team. 

Satellite family service coordination teams (n=4) involve service centers that were not 
located at operating schools. Two of these satellite service centers emphasized 
identification of family needs and referral to community resources, whereas two 
provided more direct services. 

Family service coordination teams (n=12) did not base their services around a physical 
location; rather, they featured teams of service professionals who assessed the goals 
and needs of students or families who had been identified as at risk and developed 
service strategies to address them. These programs also tended to involve more 
school personnel as project staff than did other types of programs. 

Youth service programs (n~8) addressed the health, education, and social needs of 
adolescents. Five of these programs organized themselves around a school-based or 
school-linked health clinic, whereas three were organized to make accessible to their 
adolescent clients a broad array of services to improve their health, academic 
achievements, and the likelihood of staying in school. 
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There were noticeable differences among the four types of Healthy Start programs in the 
patterns of services they provided, as differences in the educational outcomes experienced by 
their students. 



Table 7 

STUDENT GRADE PERFORMANCE (GPA), BY PROGRAM TYPE AND GOALS 



GPA before GPA after Change in GPA 
Healthy Start Healthy Start after Healthy Start 



Program Type 

School-site family resource ^ gg 2.02 .13 46 
centers 

Satellite family service centers - - - 15 

Family service coordination 2 02 2.12 .10** 103 
teams 

Youth Service Program 2.21 2.25 .04 104 

Improving Education Outcomes 
is a Program Goal 

Yes 2.08 2.16 .08* 236 

No 2.01 2.03 .02 32 



p<.05; **p<.01. 



As presented in Table 7, while students in Healthy Start programs characterized as 
school-site family resource centers experienced gains in their GPA, only those in programs 
characterized as family service coordination teams experienced a significant GPA increase 
(.10 gain; p<01). This team-based approach, more heavily involves school staff and teachers 
than other kinds of programs. These team-based programs also tended to be less family 
focused and more student focused than were family resource or satellite centers. It is not 
surprising that students in youth service programs showed the smallest gains, since, as was 
indicated earlier, students in the earlier grades showed greater improvement in their grades 
than did those in the later grades, and youth service programs primarily served secondary 
school students. 
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Healthy Start programs also varied in the types of goals they hoped to achieve. More 
than 80% of the Healthy Start programs had specified "improving students' educational 
outcomes" as being a primary goal. Students in those programs that had a stated goal of 
improving educational outcomes had even greater educational gains than did those in 
programs that did not have clearly stated educational goals (a gain of .8, p<.05), compared 
with .2, not a significant improvement). 

Linking Services to Outcomes 

Healthy Start programs provided a broad array and intensity of services to Healthy Start 
families and children. The Healthy Start legislation called for the provision of support services 
that address at least four areas of children's and families' needs; thus, virtually all programs 
offered educational, health, mental health, and social services. Most Healthy Start projects 
went beyond these four areas of service to meet a broader array of clients needs. 
Furthermore, most projects provided a large number of preventive services in addition to 
interventions. 

Academic services, especially tutoring and other academic interventions, such as 
homework clubs and peer tutoring programs, represented nearly a quarter of all services 
delivered (Figure 1). Medical health services, predominantly health screenings, accounted for 
an additional 19% of services. Basic needs provision (e.g., food, clothing, utilities assistance, 
transportation assistance, child and after-school care) and mental healthy services (primarily 
individual and group therapy) accounted for 14 and 13% of all services, respectively. Health 
education, which incorporated a wide range of topics form health enhancement (e.g., weight 
and stress reduction) to substance abuse prevention, accounted for 10% of the services. 
Family functioning services (primarily parenting education) accounted for an additional 5%. 
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Other services 




Medical health 5% 
services 

(n=l 06,644) 19% 



FIGURE 1 DISTRIBUTION OF RECORDED SERVICES DELIVERED 



What is the relationship between receiving Healthy Start services and changes in 
education outcomes? 

The conceptual framework, presented in Figure 2, has guided the Healthy Start 
evaluation form its initial design, and reflects what was known and hypothesized form existing 
research about comprehensive integrated school linked services. The framework was the 
basis for determining the components of the project and the contents of the data to be 
collected in each component. It is also the foundation for the analyses whose findings are 
reported here. 



Li 

18 



STATE CONTEXT [§) 

Collaborative 
Goals/mission 
Membership 
Structure 
Leadership 
History 

Funding arrangements 
Barriers/obstacles 
Status of law/policy/regulations 
Related policies/programs 
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COUNTY/LOCAL 
CONTEXT 

Collaborative 
Goals/mission 
Membership 
Stiucture 
Leadership 
History 

Role of parents/students 
Processes 
Planning 
Decision-making 
Information sharing 
Working relationships 
Barriers/obstacles 
Local support 
Existing service programs/ 
agencies 



SERVICE DELIVERY Pj 

Comprehensiveness 
Number of providers 
Types of providers 
Types of staff 

Integration 
Staff training 
Case management 
Integration with schools 
Information sharing 

Accessibility 
Location 

Hours of operation 
Family focus 
Prevention focus 
Cultural appropriateness 



SERVICES PROVIDED Is 

Type/combinations 
Number 
Intensity 
Breadth 

Clients served/not served 
Child or family focused 



FAMILY/CHILD OUTCOMES 



IF 



Family unit 
Basic needs 
Family/functioning 

Use of community resources/entitlements 
Adult members 

Health/mental health 

Education/skills 

Employment 

Involvement in children's schooling 
Attitudes toward services/school/system 
Substance use 
Legal system involvement 
Childbearing/birth outcomes 
Student 

School attendance/performance/completion 
Health/mental health 
Attitudes toward services/school/system 
Substance use 
Legal system involvement 
Childbearing/sexual behavior 



SCHOOL COMMUNITY 
OUTCOMES 



School community 
School climate 
Aggregate school attendance/ 

performance/completion 
Aggregate school violence/ 

suspensions/pregnancies 



SCHOOL COMMUNITY/FAMILY/INDIVIDUAL CHARACTERISTICS 



School Community 

Ethnic/cuitural diversity 
Resources/economics 
Urban/surburban/rural 



Household 
Composition 
Ethnic/cultural diversity 
Strengths/needs 



Individual 
Age 
Gender 

Language preference 
Strengths/needs 



FIGURE 2 CONCEPTUAL FRAMEWORK OF THE HEALTHY START INITIATIVE 



The framework suggests that the interrelationships among Healthy Start programs and 
services and the experiences and outcomes of children and families are many and complex. 
For example, we have seen that different kinds of Healthy Start programs served different 
kinds of clients. We also have shown that different programs are associated with different 
educational outcomes. However, we cannot be sure whether those different outcomes result 
form the differences in programs, form the differences in the clients served by them, or form 
other factors. 

When factors related to outcomes are interrelated in this way, multivariate statistical 
analysis is a useful tool to disentangle the relationships between outcomes and the multiple 
factors that may contribute to them. Multivariate analysis identifies the independent 
contribution of each independent variable while holding constant the contributions of all other 
variables in the analysis. Thus, the impact of services on outcomes can be identified for 
clients that have essentially been made "statistical twins" on all other factors in the analysis. 
Here we explore the question of whether services contributed to improved educational 
performance, specifically the changes in their grade point average (GPA) that occurred form 
the period before intake to Healthy Start, to the semester or so after intake. Linear regression 
analysis is the technique used to identify contributors to change in this dependent variable. 

Healthy Start programs provided a variety of educational services in hopes of improving 
students' school performance. To identify the contribution of services to changes in students' 
GPAs, we conducted separate analyses for each of four types of educational 
services — tutoring, counseling, advocacy, and parent education — as well as one for a 
combination of the four. We also included in the analysis a variable indicating whether or not 
the program that served students had a specific goal of improving educational performance. 
Also included was the grade level of students and the GPA they had earned before intake to 
Healthy Start, to determine whether earlier results suggesting that younger students and 
poorer-performing students were showing the greatest grade improvements were confirmed 
when other differences between students were held constant. The analyses also included 
aspects of students that other research has shown to be related to variations in educational 
performance, including the ethnic background of students and whether they received public 
assistance (Food Stamps or AFDC) as an indicator of poverty. Multivariate regression 
analyses coefficients are presented in Table 8. 
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Table 8 



LINEAR REGRESSION COEFFICIENTS RELATING SERVICES 

TO CHANGES IN GPA 



Coefficients form Model Testing Relationship of GPA to: 



Change in GPA Related to: 


Tutoring 


School 
Counseling 


School 
System 
Advocacy 


Parenting 
Education 


Total # of 
Academic 
Services 


Being Latino (vs. white) 


-.20 


-.19 


-.23 


-.21 


-.20 


Being Asian (vs. white) 


.08 


.09 


.05 


.07 


.09 


Being African American 
(vs. white) 


-.06 


-.02 


-.06 


-.04 


-.03 


rACCclVIIly ArUv Ul rUUU 

Stamps 


Dfi 


07 

.Ul 


OR 


05 


.06 


Grade level in school 


-.00 


-.01 


-.00 


-.00 


-.00 


GPA before receiving Healthy 
Start services 


01** 

-.£■5 




OA** 


OA** 


OA** 


Each unit of service received 


.36 


.04 


.00 


.05 


.02 


Served by program with goal of 
improving academic 
performance 


.06 


.09 


.07 


.06 


.08 


Number of basic needs family 
has 

r2 


-.10 
.13 


-.11 
.13 


-.11 
.13 


-.11 
.13 


-.11 
.13 


n of students 


104 


104 


104 


104 


104 



When controlling for other factors in the analyses, students' ethnic background and 
grade level and whether they received public assistance were not significantly related to 
changes in GPA. However, there was a consistent pattern of Asian students doing marginally 
better than white students and Latino and African American students doing marginally less well 
than white students. Also, consistent with earlier analysis, those who were having the most 
educational problems before Healthy Start experienced the greatest educational growth; 
students' GPA before Healthy Start was significantly related to changes in GPA (p<01). 

Students in programs with a stated goal of improved school performance were 
consistently more likely to show GPA gains than were students in programs that did not have 
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an overtly stated educational objective, although differences were quite small and not 
statistically significant. Also, all measures of educational services were consistently and 
positively related to improved educational outcomes, with tutoring programs showing the 
greatest impact. However, again, differences were small and not statistically significant at this 
early stage in the Healthy Start program. Yet the pattern of effects suggests that independent 
of other factors, receiving a greater number of tutoring, counseling, or advocacy services or 
having parents who participated in greater numbers of parenting education sessions each was 
positively, though not significantly, associated with small improvements in GPA. The sum of all 
of these kinds of educational services also was marginally but positively related to small GPA 
improvements. 

Thus, the pattern of data suggests that educationally oriented services may contribute 
to small gains in school performance even after relatively short participation in those services. 
Yet the Healthy Start Initiative assumes that educational performance can be improved not just 
through addressing students' educational needs directly, but also through meeting other needs 
that present barriers to learning, such as poor health, being hungry, or experiencing family 
stresses. Analyses confirm that, controlling for other differences between them, students form 
families who had needs for food, clothing, or other such basics were less likely to experience 
gains in educational performance than students with fewer of such needs. Thus, meeting 
families' basic needs presents an opportunity to eliminate barriers to learning. Further, we 
have found a strong and significantly positive relationship between providing basic needs 
services through Healthy Start and reduction in such needs, suggesting that Healthy Start 
basic needs services were effective in reducing the problems they were intended to ameliorate 
and, thus, reducing the barriers to learning presented by deficits in such basics as food and 
clothing. 

In short, it appears that it helps to help. We are beginning to see a pattern of positive 
associations between providing a variety of services through Healthy Start programs and 
achieving positive education impacts for children and families, although impacts are small at 
this early stage and often not statistically significant when differences between students are 
controlled for. Healthy Start grantees are funded for 3 years, and the statewide evaluation will 
track their development over that time. We are currently in the process of collecting school 
performance information for more than 2,000 Healthy Start core students. Further evaluation 
will assess whether continued participation in services is associated with larger gains over 
time. 
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Appendix A 
OVERVIEW OF DATA COLLECTION 




Healthy Start Instrument/Survey Matrix 



Instrument/Survey 


Respondent Pool 


Data Collection 
Time Interval 


Number 
Collected 


Response 
Rate 


Instruments for Documenting 
Collaborative Processes: 










Phone Interview-coordinator 


All Healthy Start 
coordinators 


February 1993 


40 


100% 


Phone Interview-agencies 


A sample of public agency 
personnel involved in the 
collaborative (4-6 per site) 


February 1993 


151 


90% 


Collaborative Membership List 


All Healthy Start 
coordinators 


March 1993 
October 1994 


37 


93% 


Principal Survey 


Principals of all schools 
affiliated with Healthy 

Qtart nrnnramc 
Old) I piuyiaiiio 


Spring 1993 
Spring 1995 


82 


65% 


School Staff Survey 


Random sample of school 
personnel in schools 
affiliated with Healthy 
Start programs 


Spring 1993 
Spring 1995 


509 


70% 


Healthy Start Project Staff 
Survey 


All 1 i fc 4 i /*x * j _ . _ * _ _ * 

All Healthy Start project 
staff members 


Spring 1993 
Spring 1995 


249 


7CO/ 
/DVO 


Coordinator Survey 


All Healthy Start 
coordinators 


Spring 1993 
Spring 1994 
Spring 1995 


36 


90% 


Public Agency Survey 


All remaining public 
agency personnel 
involved in the 
collaborative who were 
not interviewed by phone 


Spring 1993 
Spring 1995 


68 


68% 



Healthy Start Instrument/Survey Matrix (continued) 



Instrument/Survev 

II 1911 M11IVIIW W Ul VvJ 


Recnonrient 

IwOpvl lUvl 11 


Data Collection 

Tim A IntAwal 


Number 


Response 

Rata 


Instruments for Documenting 

wCI VIUC9 










Encounter Log/Form 


Healthy Start project staff 


Every time a project staff 
memuer nas contact witn 
a client 


15486 


Not 
sppiicaDie 


Referral Follow-up Log/Form 


Healthy Start project staff 


Every time a project staff 
member follows up on an 
external referral s/he 
made 


802 


Not 
applicable 


Attendance Sheet 


Healthy Start project staff 


Every time a small group 
event occurs 


2317 


Not 
applicable 


Event Description Form 


Healthy Start project staff 


Every time a large 
service event occurs 


662 


Not 
applicable 




Instruments for Documenting 
Client Outcomes 










Intake/Assessment Form 


Healthy Start project staff 


Each time a new core 
family or youth is enrolled 
for services 


1796 


Not 
applicable 


Follow-up Assessment Form 


Healthy Start project staff 


Approximately 6 months 
after corresponding 
intake assessments 


400 


Not 
applicable 


Parent/Caregiver Questionnaire 


Self administered by 
parents 


At intake assessment and 
all follow-up assessments 


486 


Not 
applicable 


Student Questionnaire 


Self administered by 
secondary school youth 


At intake assessment and 
all follow-up assessments 


369 


Not 
applicable 



Healthy Start Instrument/Survey Matrix (concluded) 



Instrument/Survey 


Respondent 


Data Collection 
Time Interval 


Number 
Collected 


Response 
Rate 


Instruments for Documenting 
School-level Outcomes 










Student Abstracts 


Healthy Start project staff 
or school personnel 


Winter 1994 
Spring 1995 


545 


81% 


School Background Survey 


Principals of all schools 
affiliated with Healthy 
Start programs 


Fall 1993 
Spring 1995 


100 


79% 


Average Daily Attendance 
Survey 


Healthy Start coordinators 
and Budget and Finance 
directors 


Winter 1994 
Spring 1995 


108 


85% 
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OTHER REPORTS BASED ON THE HEALTHY START EVALUATION 
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Reports and Papers Available 
from the Statewide Evaluation of California's 
Healthy Start Initiative 



Order 
No. 


Title 


Unit 
Price 


Quantity 


Total Price 


197 


Collaborative Planning for School-Linked 
Services: An Evaluation of California's Healthy 
Start Planning Grants, by Mary Wagner, 
February 1994 (70 pages) 


$10.00 
+ $.83 

tax 
within 

Calif. 






198 


Implementing School-Linked Services: A 
Process Evaluation of the First Year of 
California's Healthy Start Initiative, by Mary 
Wagner, Debra Shaver, Lynn Newman, Marjorie 

\A/a/«hfrlar Cr\r\*3 l^allfiii SL Chart CXd\d r\ P- O Kn lorv 

wecnsier, riona rveuey, o» onan vsuicmi, reuiuaiy 
1994 (94 pages) 


$12.00 
+ $.99 

tax 
within 
Calif. 






199 


A Healthy Start for California's Children and 
Families: Early Findings from a Statewide 
Evaluation of School-Linked Services, by Mary 
Wagner, Shari Golan, Debra Shaver, Lynn 
Newman, n/iarjone wecnsier, o» riona r\euey, 
June 1994 (158 pages) 


$18.00 
+ $1.49 

tax 
within 

Calif. 






205 


Developments in Collaborative Planning for 
School-Linked Services: Findings from the 
Second-Year Evaluation of Planning Grants in 
California's Healthy Start Initiative, by Shari 
Golan, Fiona Kelley, and Mary Wagner, February 
1995 (88 pages) 


$12.00 
+ $.99 

tax 
within 

Calif. 






Add $10 handling fee if payment is not enclosed: 




Subtotal: 




(In CA add appropriate state and local sales tax) Tax: 




Shipping and handling within the U.S., Canada, and Mexico are 
included. Payment must accompany order. If payment not 
included, $10.00 service charge for invoicing. 


Total: 





Ship to: 

Name 

Affiliation 

Address 



City, State, ZIP 



Mail order form and payment (payable to the order of SRI International) to: 

SRI International 

Room BS178 

333 Ravenswood Avenue 

Menlo Park, CA 94025-3493 

Telephone: (415)859-5109 

Fax: (415)859-2861 

Note: Our federal tax ID number is 94-1 160950 

Our California tax ID number is SR BHA 20-141387 
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